Account Pool: model-family-aware routing and guarded account_uuid rewrite - #3053
Merged
Conversation
This was referenced Sep 4, 2026
SawyerHood
added a commit
that referenced
this pull request
Sep 4, 2026
…3054) Stack layer 2 (Codex pool track). Prerequisite: #3053 below it. Core-only: adds plugin WebSocket routes so a provider pool can terminate Codex's WebSocket-first transport at the hub. ## Human comments ## What was wrong Plugin HTTP routes had no WebSocket upgrade surface. Codex derives its WebSocket URL from the configured HTTP base URL and tries the Responses WebSocket protocol first, so an Account Pool proxy mounted in a plugin's existing `/http/` namespace could serve HTTP but could not accept the corresponding upgrade. Codex therefore had to wait through failed WebSocket attempts before falling back to HTTP SSE. ## What changed - Added `bb.http.experimental_websocket(path, handler, opts)` and its experimental socket, context, handler, and lifecycle types. - Mounted exact-path upgrades in the existing `/api/v1/plugins/<id>/http/` namespace while preserving ordinary HTTP handling on the same path. - Applied the existing `local`, `token`, and `none` auth modes to upgrade requests, including header and query-token authentication. - Isolated connection and event callbacks through the plugin invocation boundary, serialized async callbacks per socket, normalized binary frames to `Uint8Array`, and closed sockets with code 1012 when their plugin generation reloads or is disabled. - Added fake-host registration and socket-driving support, public surface/audit entries, Plugin Guide references, and bumped the Plugin SDK to 0.4.46. There is no host-daemon wire contract change, so `HOST_DAEMON_PROTOCOL_VERSION` does not need a bump. The existing machine-auth proxy already forwards authenticated WebSocket upgrades and retains its focused regression coverage. ## How you verified The new server and fake-host regressions fail on `origin/main` because the WebSocket API, registry, dispatch, and test driver do not exist there. They pass on this branch and cover exact-path upgrade dispatch, plain-GET behavior, HTTP/WebSocket coexistence, all auth modes, text and binary frames, callback failure isolation, and code-1012 reload shutdown. - `pnpm exec turbo run typecheck test --filter=@bb/server --filter=@get-bb/plugin-sdk --filter=@bb/plugin-api-map --continue` — 13/13 tasks passed; SDK 232 tests, API map 72 tests, server 2,209 tests; all typechecks passed. - `pnpm exec turbo run test typecheck --filter=@get-bb/plugin-sdk --filter=@bb/plugin-api-map --filter=@bb/app --filter=bb-plugin-plugin-api-docs --continue` — 11/11 tasks passed; app 3,872 tests passed and 4 skipped. - `pnpm exec turbo run test --filter=@bb/host-daemon -- --run src/machine-auth-proxy.test.ts` — 8/8 tests passed, including WebSocket upgrade forwarding with machine authentication. - `bb plugin build plugins/plugin-api-docs` — production build passed. - `node .github/workflows/check-plugin-sdk-version.mjs` — confirmed the public surface changed with a version bump. - Live dev app: installed a throwaway plugin with `auth: "none"`, connected to `/api/v1/plugins/websocket-e2e/http/v1/echo`, and echoed `alpha`, `beta`, and binary `[0, 127, 255]` in order. Tracks BB-93 (layer A). > AGENT GENERATED
SawyerHood
added a commit
that referenced
this pull request
Sep 4, 2026
) Stack layer 3 (Codex pool track). Prerequisites: #3053 and #3054 below it. Single-plugin design: Codex joins the existing Account Pool plugin as a provider adapter. ## Human comments ## What was wrong Account Pool was coupled to Claude credentials, request rewriting, quota headers, and refresh behavior, so it could not select or maintain Codex OAuth accounts. Codex also attempts the Responses WebSocket protocol before HTTP SSE, while the Codex bridge launched `codex app-server` without applying provider-contributed per-thread base URL and hub authentication values. Pointing Codex at the hub therefore required both a provider-neutral pool core inside the plugin and a session-aware app-server launch path. ## What changed - Added an internal provider-adapter contract in `plugins/account-pool` and preserved the existing Claude behavior behind its adapter. - Added Codex OAuth import from `~/.codex/auth.json`, serialized pre-expiry refresh with write-back, ChatGPT account identity storage, and provider-scoped account selection. - Added Codex request/header rewriting for ChatGPT's Codex backend and parsed `x-codex-primary-*` / `x-codex-secondary-*` rate-limit observations and quota rejections. - Added authenticated HTTP `POST /v1/responses`, `GET /v1/models`, and WebSocket `/v1/responses` routes in the existing Account Pool `/http/` namespace. - Implemented Codex's downstream envelope session behavior: local prewarm completion, incremental input expansion after a completed response, unknown-id failure plus close 1011, upstream HTTPS SSE forwarding as one JSON event per WebSocket frame, and rotation after quota rejection. - Made each WebSocket forward independently abortable. A downstream close now cancels the hub request signal and upstream fetch/body read, releases the account's in-flight slot, drops queued frames, and guards every send against a closing or closed socket. - Extended the single `bb pool` CLI and Account Pool settings section with Codex import, provider badges, shared machine tokens, Codex environment and health contributions, and provider-aware status. - Made the Codex bridge consume the resolved per-session pool environment, append in-memory `-c openai_base_url=...` configuration, add an environment-backed hub header without putting the token in process arguments, and rebuild a session when pool routing or token identity changes. - Updated the configuration reference, generated Guide source, and builtin bb CLI skill. The original two-plugin extraction plan was intentionally dropped after the scope decision to keep both providers in `plugins/account-pool`. There is no server/host-daemon wire contract change, so `HOST_DAEMON_PROTOCOL_VERSION` does not need a bump. There is no new public Plugin SDK surface in this layer. ## Not in this PR - Keeping `model_provider="openai"` while adding the machine-token header is not possible with Codex CLI 0.153.2: a live app-server launch rejects `model_providers.openai` because built-in provider IDs are reserved and cannot be overridden. The bridge therefore retains the custom `bb-account-pool` provider ID. That changes the provider recorded in newly pooled rollouts and may affect Codex's provider-keyed session grouping, remote compaction, and remote-control behavior. Compatibility for existing sessions was verified live: a rollout created with `model_provider: "openai"` completed a direct turn, Account Pool was enabled, and the same rollout resumed and completed a pooled WebSocket turn. ## How you verified The new focused regressions fail on the layer-A base because Codex is not an Account Pool provider there and the bridge ignores per-session pool configuration. They now cover import parsing, refresh persistence, HTTP credential/account-id rewrite, provider-scoped selection, Codex quota parsing and rotation, WebSocket authentication/prewarm/incremental/unknown-id/session streaming behavior, provider env/health, UI import and badges, and token-free app-server launch arguments. - `pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool --filter=bb-plugin-provider-codex --continue --force` — 8/8 tasks passed; Account Pool 41/41 tests and Codex provider 261/261 tests passed; both typechecks passed. The added regression closes a downstream WebSocket during a non-terminating SSE response and proves the upstream abort signal fires, the account's in-flight count returns to zero, and no post-close frame is sent. - `pnpm exec turbo run test typecheck --filter=@bb/server --filter=@bb/app --filter=@bb/templates --filter=@bb/cli --continue` — 15/15 tasks passed; server 2,209, app 3,872, templates 43, and CLI 530 tests passed; 4 app tests skipped; all selected typechecks passed. - `bb plugin build plugins/account-pool` — production server and app bundles built successfully. - `node scripts/check-provider-literal-ratchet.mjs` — passed with all core provider literals allowlisted. - `node .github/workflows/check-plugin-sdk-version.mjs` — passed against the layer-A SDK surface/version state. - `git diff --check` — passed. - Live dev app with Codex CLI 0.153.2 and this machine's imported account: a pooled WebSocket turn returned `BB93_POOL_WS_OK`, emitted the Account Pool WebSocket transport marker, and updated five-hour utilization to 0.4% without fallback/retry noise. The resolved-env card showed the Account Pool base URL and masked token. After `bb pool bypass`, a direct turn returned `BB93_DIRECT_OK` while the pool observation timestamp stayed unchanged. Settings and timeline screenshots were inspected and attached to BB-93. - Review E2E with Codex CLI 0.153.2: the attempted built-in `openai` provider override failed with the reserved-provider validation error. With Account Pool disabled, rollout `01a06b6b-a3a4-7de2-9d07-d98af7bc5c90` recorded `model_provider: "openai"` and returned `BB93_PREPOOL_OPENAI_OK`; after enabling Account Pool, that same rollout returned `BB93_RESUMED_THROUGH_POOL_OK` over the plugin WebSocket route and advanced pool utilization from 0.40% to 0.41% with no account error. ## Review fixes - Malformed or otherwise failing Codex SSE forwarding now aborts the hub request and cancels the response reader on every non-EOF exit, immediately releasing the selected account's in-flight slot. - Added `releases a Codex request when an upstream SSE event is malformed`, which sends `data: {\n\n` on a stream that stays open and proves `response.failed` plus zero in flight through `bb pool status --json`. - `pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool --continue` passes with 42/42 tests and all selected typechecks. Tracks BB-93 (layer B). > AGENT GENERATED
ymichael
added a commit
that referenced
this pull request
Sep 8, 2026
## Human comments ## What was wrong [PR #3053](#3053) correctly switched accounts when the first upstream response declared a model-family limit, but its short same-account retry used a separate response path. If the first response was an ordinary short 429 and the retry then declared a family limit, that second path returned the 429 without trying another eligible account. [PR #3117](#3117) later consolidated attempts into one loop and incidentally fixed the root cause on current `main`, but no regression test covered the original two-stage response sequence. ## What changed Added a shipped Account Pooler plugin test using the Plugin SDK fake-host harness. It drives the exact sequence `sk-one` ordinary short 429, `sk-one` family-limit 429, then `sk-two` success, and asserts the client receives HTTP 200. There is no production, wire-contract, CLI, guide, documentation, credential, quota, or unrelated-provider change. ## How you verified - Red on the [original landing](c2d5bd3): HTTP 429 with attempts `sk-one, sk-one`; `sk-two` was not tried. - Green on current `main`: the focused harness test passed with `sk-one, sk-one, sk-two` and HTTP 200. - `pnpm exec turbo run test typecheck build --filter=bb-plugin-account-pool --concurrency=1 --continue --force` — 6 tasks passed; 10 test files and 262 tests passed; typecheck passed. - `pnpm exec oxfmt --check plugins/account-pool/src/server.test.ts` and `git diff --check origin/main...HEAD` passed. Part of #1552 > AGENT GENERATED
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Standalone follow-up to the Account Pool stack (#3035, #3036, #3042, #3049); based on main.
Human comments
What was wrong
Account Pool selected accounts using only shared 5-hour and 7-day quota observations, even though Anthropic applies separate weekly limits to model families such as Fable. Scoped
7d_<bucket>headers were recorded as opaque diagnostics and never governed selection, the OAuth usage endpoint was not polled, and Claude Code request metadata could retain an account UUID belonging to a different OAuth account after routing. Part of #1552.What changed
user_*_account_<uuid>_session_*metadata forms. Unrecognized, absent, non-JSON, or UUID-less requests remain byte-identical.bb pool account listand updated the CLI skill, plugin guide, and configuration docs.How you verified
pnpm exec turbo run typecheck test --filter=bb-plugin-account-pool --continue --force— 6 tasks passed; 4 test files and 35 tests passed. The new routing, scoped-429, refresh, migration, and UUID tests fail against the prior implementation and pass with this change.pnpm exec turbo run typecheck test --filter=@bb/templates --continue --force— 7 tasks passed; 7 test files and 43 tests passed.pnpm exec turbo run typecheck --filter=@bb/server --continue— 5 tasks passed.pnpm --silent bb plugin build plugins/account-pool— server and app bundles built successfully.git diff --check origin/main...HEAD— passed after rebasing onto currentorigin/main.switchThreshold=0.70: Account A had shared 7d at 43% but Fable at 80%; Account B had Fable at 7%. A realclaude-fable-5-1turn completed through Account B, then a realclaude-sonnet-5turn completed through Account A, proving family-only exhaustion does not disable the account globally. A redacted shape probe on a further real turn foundmetadata.user_idpresent without a recognized account UUID component, so the production path correctly preserved it byte-for-byte. The probe was removed, both imported test accounts were removed from the isolated store, and the dev app was stopped. Evidence is attached to BB-94.Part of #1552